When law enforcement officers talk about the "most effective"caliber bullet or the "best" combat handgun on the street,emotions run high and opinions vary. This can be expected, sincethese topics have caused considerable debate for years.
But what of the firearms expert who is tasked with theresponsibility of selecting ammunition and firearms for adepartment? What are the crucial issues that should beconsidered? Where should testing begin? What needs to beaddressed in order to conduct a fair and impartial ammunition andfirearms selection program?
The FBI Academy's Institutional Research and Development Unit(IRDU) provides consultation primarily to the FBI's TrainingDivision personnel regarding research methodology, evaluationand statistical analysis. This article provides an introductionto research design and statistical analysis with regard toammunition selection. It is intended to assist firearmspersonnel in designing an ammunition research project andanalyzing the results.
The topics addressed include (1) research design, (2) criteriafor selecting ammunition, (3) rater bias, and (4) statisticalanalyses. Throughout the article emphasis is placed onunderstanding the logic of the various elements of a researchproject.
DESIGN OF THE RESEARCH
Kerlinger, a research methodologist, indicates that researchdesign is the structure, plan or strategy developed to obtainresults from a research project. "Research designs are inventedto enable the researcher to answer research questions as validly,objectively, accurately, and economically as possible."(1)
In designing any ammunition selection study, the first step is todetermine the comparisons to be made. For example, is thepurpose of the study to compare the same caliber bulletperformance for ammunition made by different companies or to
compare the performance of the same caliber bullet in handgunsproduced by different manufacturers?
The following research design is used throughout this article asa convenient example; three different calibers are compared onperformance measures of penetration, expansion and weight in avariety of target simulants (targets). Examples of targets aregelatin blocks to simulate human tissue, sheets of metal toresemble the properties of an automobile door, automobilewindshield glass held at a given angle, and so on.
"Internal validity" and "external validity" are two majorcriteria by which any research design is judged. Internalvalidity, for the example shown above, is the extent to whichdifferences in penetration, expansion and weight can beattributed to differences in the physical characteristics of thecalibers rather than to other influences or conditions. Externalvalidity is the extent to which similar differences inperformance would generalize to other ammunition, conditions orsettings. The ideal would be to maximize both internal andexternal validity. However, the importance of maximizinginternal validity, that is, controlling unwanted influences, by
necessity, often limits external validity.
Internal Validity
Internal validity is extremely important in any ammunitionselection study; if the research is internally valid, then thereis a high probability that the differences in caliber performanceare caused by the different sizes of the calibers. Internalvalidity is synonymous with control over unwanted influences. For ammunition selection studies, the unwanted influences thatmust be controlled or held constant would include environmentalconditions, physical/human conditions, and target simulants.
Environmental conditions-In an indoor range, environmentalconditions for firing ammunition can be easily controlled. Shooting should take place where temperature, weather, light andnoise are kept fairly constant. Without an indoor range, keepingthese conditions constant is extremely difficult.
Physical/human conditions⌐⌐Many other physical and humaninfluences can affect a study. Some of these influences can bedetermined; others cannot. The best way to control unwantedinfluences is to simultaneously set up test barrels, one for eachcaliber to be tested, and randomly determine the order in whichthe test barrels are fired. (A table of random numbers can beused to determine the order.) For example, a researcher whofires one caliber all morning and then fires a different caliberthroughout the afternoon might have measurements influenced bythe fatigue of late afternoon shooting and thereforeunintentionally record measurement results favoring the calibershot in the morning.
Other variables are not controlled by random ordering for firingthe different calibers. For example, if test barrels are not ofequal length, firing them in random order would not compensatefor these differences. Using test barrels of unequal length willaffect not only the velocity but also the extent of penetration. Therefore, if unequal length test barrels are used, additionalresearch is necessary to determine the öextentò of the differencesamong the calibers tested, which adds greatly to the complexityof the research.
Targets-Whether one type of target or a variety of targets areused in the study, controlling the variations in theconstruction of these targets is critical and can be done byrandomly distributing targets (again using the random numberstable) of a given type across calibers. For example, if a batchof gelatin blocks is not mixed thoroughly and blocks with greaterdensity are used with only one caliber, then any differences inpenetration, expansion or weight for the different calibers couldbe partially or fully caused by the consistency of the gelatinblocks.
Since gelatin blocks are used both as stand-alones and behindother targets, two other controls are suggested. First, becausegelatin blocks can deteriorate easily, care must be taken topreserve their integrity. Gelatin blocks should be stored ininsulated coolers prior to use and should be checked by measuringtheir temperature before being used for targets. Second, analready-penetrated gelatin block should not be used again as atarget. The trauma from the first round's impact may disturbthe consistency of the gelatin and affect the measurement ofpenetration from later rounds fired into it.
External Validity
After maximizing internal validity, the reseaercher must alsoplan for external validity so that the results can be generalizedbeyond the bullets used in the study. There are many conditionsunder which results may be generalized; no study can accomplishall of them. However, it's important to know what theseconditions are since the generalizations that cannot be made setthe limitations of the study.
External validity is the extent to which any difference inperformance among the calibers can be generalized to (1) a largerpopulation, such as other lots of ammunition of the same calibermade by the same manufacturer; (2) different populations, such asother ammunition of the same caliber made by differentmanufacturers; (3) "real-life" targets that the study targetspurport to "simulate"; and (4) other conditions and settings.
How can a researcher determine if the results of a study can begeneralized to a larger population of other same caliber bulletsfrom the same manufacturer? If the bullets in a study are arandom sample from this larger population of bullets, the
bullets are representative of that population. This means thatany sample of the same caliber bullets from this population canbe expected to produce similar results.
How can the results be generalized to other conditions orsettings? One way is to build important conditions into theresearch design. When the study at the beginning of this articlewas designed to compare the performance of different calibers ina variety of targets, we d
